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Abstract — Demand side management has gained increasing 
importance as the penetration of renewable energy grows. Based 
on a Markov jump process modelling of a group of thermostatic 
loads, this paper proposes a two level feedback system design be- 
tween the independent system operator (ISO) and the regulation 
service provider such that two objectives are achieved: (1) the ISO 
can optimally dispatch regulation signals to multiple providers 
in real time in order to reduce the requirement for expensive 
spinning reserves, and (2) each regulation provider can control its 
thermostatic loads to respond the ISO signal. It is also shown that 
the amount of regulation service that can be provided is implicitly 
restricted by a few fundamental parameters of the provider itself, 
such as the allowable set point choice and its thermal constant. 
An interesting finding is that the regulation provider's ability 
to provide a large amount of long term accumulated regulation 
and short term signal tracking restrict each other. Simulation 
results are presented to verify and illustrate the performance of 
the proposed framework. 



I. Introduction 

Wind energy, known as the most widely used form of 
clean and renewable generation resource in power systems, 
is increasing its penetration around the world. Europe, Asia, 
and North America will have an exponential yearly increase 
in the amount of added wind power capacity by 2020 ||T|. 
The increase in wind energy makes the generation side less 
controllable due to the intermittent feature of wind. As a 
consequence, building level demand side management (DSM) 
for thermostatic load becomes a crucial part that can be 
achieved generally through either a price-based signalling 
control El -151 or a non-priced direct load control (DLC). 
Compared with pricing mechanism, the operator in DLC can 
have better understanding of the loads' response in order to 
provide short time scale regulation services |6|. 

DLC approaches on thermostatic loads have been studied 
extensively during the past decade for difference purposes, 
including load shifting, load smoothing, and load following. 
A primitive investigation on load shifting has been studied 
with a state queuing model to illustrate the effect of set-point 
change |7|. It is shown that even a simple step change will 
result in a complicated load profile due to the instantly change 
of loads' status. Taking into account both the appliances 
duty cycle as well as probabilistic consumer behaviour, a 
steady state appliances population distribution during an load 
shifting demand response is analysed in (H. Results shows 
that the steady state distribution exists and may not be as 
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simple as uniform. More recently, the possibility to integrate 
DLC into load smoothing or following is investigated. The 
notion of packetized DLC is proposed in f9| to smooth 
electricity consumption with shortened appliances duty cycle. 
Load following can also be found in |[Tq1| - |[T21 where various 
of control methods including discrete time directly on/off 
actuation control, set point shifting, as well as model based 
predictive control are used. Results in these papers show that 
loads, by acting as both a positive and negative generation 
source, is a promising resource to response against the load 
request from the ISO to maintain supply demand balance. 

While the study of the DLC itself seems to be comprehen- 
sive, there are two issues remain to be answered. The first one 
is the limitation of regulation provision, namely what is the 
maximum regulation can be provided by a specific provider 
given its implicit features such as population of appliances 
pool, parameters in its own model. The second one is about 
the information exchange between the ISO and the providers. 
Instead of the one way command-like signalling, can we 
design another framework such that the ISO can get feedback 
information from providers? If so, how beneficial is it to the 
real time operation? 

The objective of this paper is to answer these questions by 
developing a two layer feedback control system. We focus 
primarily on smart buildings that are operated by a central 
operator who has the authority to shift the set point of 
thermostatic appliances within the building. Consumers are as- 
sumed to authorize the operator to control their set point once 
they provide the allowable set point range. It will be shown 
that the operator in the lower feedback layer can design a 
feedback control law to track the ISO regulation signal within 
a certain ramping rate. The higher feedback loop is designed 
between the ISO and buildings where the former dispatches 
real time signals and the latter send back information to 
characterize their instant-by-instant capability to respond. We 
also derive analytically the maximum possible regulation that 
can be provided from buildings based on two considerations 
- the capability to provide long term accumulated and short 
term (one step) ramping regulation. It is further shown that 
two fundamental parameters determining the capability are 
the building thermal constant r and appliances temperature 
gain Tg. Large r enables the building to provide larger long 
term, but smaller short term regulation service. Large Tg 
performs oppositely. The overall two level system has a good 
performance in real time operation as it reduces the amount 
of spinning reserves required from high-cost generators. 

The paper proceeds as follows. Section |Il| develops the 
state space model for which the feedback system is designed. 



Section III gives the overall design of the two layer feedback 
system in which we first introduce the lower layer controller 



with feedback Hnearization, followed by the investigation of 
maximum possible regulation provision, and then the higher 
level feedback design that includes a quadratic program to 
optimally dispatch real time regulation to individual buildings. 
Section |IV] discusses an observer design to asymptotically 
reduce the estimation error for unobservable states. Simulation 
results are given in secti on [V| to show the performance of the 
proposed system. Section | VI| concludes the paper and proposes 
future work. 

II. Markov Jump Thermal Process Modelling 

We model the system as a continuous time, discrete state 
Markov jump process where a state i is defined as a pair of 
temperature value and thermostat value {T^, on(off)}. Previous 
literature has considered similar state definitions - for example 
171, 1 8 1, |10|. The departure of our modelling approach from 
previous ones is that we consider state in a Markov setting 
and establish the implicit relations between thermal processes 
and the Markov jump processes. The Markov jump process 
model in this section is the fundamental model that we based 
to design the two level feedback system and to derive the 
limitation of providing regulation service. 

Denote the fixed width comfort band around the set point as 
[Tmin, Tmax] and discrctizc temperature in the band into bins of 
width S, then the number of temperature bins is = Aband/^^- 
We say that an appliances is in state z for z = 1, . . . , A/" if 
Ti G [Tmin -\- {i — l)S^ Tmin -\- iS] with status off, and i for i = 
7V+1, . . . , 27V if G [T^,,-{i-N-2)S,T^,,-{i-N-l)S] 
with status on. 

First let our control u = and solve for the Markov 
transition rate of the uncontrolled thermal process. Denote a 
the transition rate when the thermostat is off, and (3 the rate 
when the thermostat is on; see Fig. 1(a). In the duty off process, 
the probability that an appliances will be in state i at time t-\-l 
given in state j at time t is given by. 



aAt + o{At) if z = j + 1 
otherwise, 



(1) 



where o{At) means At ~ ^' above equation 

relates transition probability and transition rate for small At. 
It can be interpreted that the transition probability between 
adjacent states linearly increases with small At. 

From a thermal point of view, the temperature rise AT 
within a small At is proportional to its warming rate Toff, 



AT = roff At. 



(2) 



Assuming that an appliance's actual temperature is uniformly 
distributed among its bins, then the probability that the state 
transits is equal to the probability that the appliance tempera- 
ture is in the dotted area in Fig. 1(b), namely appliances in the 
dotted area change their state from i to i + 1, and appliances 
outside remain in the same state. 

The probability of being in the dotted area is given by, 

roffAt AbandAt NAt 




Fig. L Markov jump process modelling, (a) Markov chain transition rate 
diagram, (b) Transition from state i to i + 1. With temperature rise AT 
(length of arrow), appliances whose temperatures inside the dotted area (A, 
C, E) change state and outside (B, D) remain in the same state. 



where the second equality follows from the relation between 
warming rate and duty cycle. Since = Pdot, comparing 

([T]) with ^ we have a = N/toff. Similarly, /3 = A'/ton for 
the duty off process. 

When control is applied to shift the set point at a rate of 
Tset (the unit of rset is the same as Ton or Toff), there is also 
a transition between adjacent states within At. Intuitively, 
when we change the set point, the absolute temperature in 
each individual room does not change instantly, but its relative 
position in the comfort band changes. When the set point rises, 
namely rset > 0, we can show that the resulting transition rate 
is u = The transition rate is the same with rset < 0. The 

combined transition rate by the thermal process and set point 
shifting process is the sum of these two individual processes 
as shown in Fig. 1(a). Note that the allowable control set is 
= {u\ — 13 < u < a} io maintain a non-negative Markov 
rate. When u > a or u < —l3,\hQ system fails to be a Markov 
chain. 

Similar to ([T]), we can write the transition probability for 
the duty off process with non-zero control 



(a - u)At + o{At) if i = j + 1 
Pi^j = \ I - [a- u)At + o{At) if i = j 

otherwise. 



(4) 



Let x{t) be a vector whose i^^ component is the probability 
that an appliance is found in the i^^ state in the Markov chain. 
The dynamics of Xi{t) for i = 2, . . . , A" is given by, 

Xi{t^At) = Pi,iXi{t) ^ pi^i-iXi-i{t), (5) 

Substitute ^ into ([5} we will have, 

Xi{t^At)-Xi{t) = {a-u)At[xi-i{t)-Xi{t)]^o{At). (6) 

Dividing At on both sides and taking At to yields the 
dynamics for the i^^ component in the duty off process. 



Xi{t) = {a - u)[xi-i{t) - Xi{t)], 



(7) 



for z = 2, . . . , A^. Similarly for the duty on process i = N - 
2,...,2A-, 



Pdot 



^off^^ 



(3) 



off 



Xi{t) = (/3 + u)[Xi-i{t) - Xi{t)], 



(8) 



and reflection boundaries z = 1, + 1, 

^l(^) = -{(^ + u)xi{t) + (/3 + u)x2N{t), 

x^+i{t) = {a - u)x^{t) - (/3 + u)x^+i{t). 
From ^ to (|9]), the overaU dynamics of x{t) is given by, 



(9) 



where 



x{t) = [A^ Bu{t)]x{t), 

-a 
a 

'•• -a ''. 



(10) 
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The output of the system, namely the aggregated consump- 
tion, is 

y{t) = Cx{t), (11) 
where C = [0, . . . , 0, A^c, • • • , ^c], and is the total number 

N N 

of appliances in the pool. 



III. Two Level Feedback System Design 

Based on the state space model developed in section 
we are able to design our two level feedback system as 
shown in Fig. 2. In the building level feedback loop, we design 
a state feedback law such that the building can track the 
required regulation signal. The higher level feedback is from 
each individual building to the ISO where the former send 
their information to the latter to characterize the building's 
capability of responding to regulation signals. The ISO, after 
receiving all information from each participant, dispatches real 
time regulation signals after solving an optimization problem. 
The next sub-section will shortly discuss the building level 
controller design by feedback linearzation. 

A. Building Level Feedback Linearization Design 

To solve the building level controller design for the non- 
linear system described in (10) and ([TT]), the feedback lin- 
earization method can be readily applied |13|. Note that the 
system has relative degree 1 as the first derivative of y. 



y = Cxit) = C[A + Bu{t)]x(t), 
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Fig. 2. Two level feedback system where the lower feedback designs the 
individual tracking controller for a given regulation signal, and the higher 
feedback enables the communication between the ISO and the regulation 
providers such that information is exchanged to reach better real time 
regulation signal dispatch. 



depends on the control u if 

x(t) e {x{t) e R^^\CBx{t) ^ 0}, 

G {x{t) e R^^\x^{t) + X2N{t) > 0}. 

This means the number of controllable appliances around the 
comfort band boundary is positive, so shifting the set point can 
change the aggregated consumption. The following change of 
variables, 

01 (x) 



T{x) 



Cx 



(13) 



yields the internal-external state space representation. 



y 



(14) 



C[A + Bu(t)]x{t) 

where r]{t) stands for the internal and ^{t) the external states 
fT3]. Then the output becomes the external variable ^{t). 

To design a controller for a relative degree 1 system such 
that lim ^{t) — R{t) = where R{t) is the required amount 

t^oo 

of regulation needs to be provided, we need R{t)^ R{t) to be 
available and bounded for allt > 0. This is obtainable since 
the ISO regulation requirement is bounded: R{t) G [—Rr^Rr] 
(Rj. is maximum regulation sold to the market), and R{t) is 
updated every 4 to 6 seconds prior to consumption. The time 
derivative of the tracking error, 

e{t) = i{t) - R{t) = C[A + Bu{t)]x{t) - R{t), (15) 

will asymptotically approach if u{t) is chosen such that 

e{t) =Ke{t),K>0. (16) 

The controller 

CAx{t) 1 



u{t) 



-K{at)-R{t))^R{t)] (17) 



CBx{t) CBx{ty 

satisfies the requirement because substituting ([T7| into ([15]) 
yields (16). For the discrete signal R{t), we can calculate the 



(12) 



first derivative of R{t) as R{t) = ^ ^| — 



B. Long and Short Term Regulation Provision 

In this section we proceed to answer the first question 
proposed in section [l| namely what is the regulation service 
limitation that a building can provide given its model param- 
eters and user specified settings. This question has two parts: 

(1) in the long term, we need the accumulated shift of set 
point to be within the allowable range to provide reserve, and 

(2) in the short term, we need the signalling change to be 
within bounds such that the controller can track it. The first 
proposition will discuss the limitation on long term regulation. 

Proposition.! For a given number of appliances and 
allowable set point range [Tset - ^Aset,Tset + ^Aset], the 
accumulated amount of reserve that a building can provide 
up to time t is given by 



The third equation is based on the relation between transition 
rate and duty cycle. The last equation says that the sum of 
cooling and warming rate is the rate that the cooling system 



generates because roff = ramb and 



' app 



ramb, see ||9l. 



^amb is the warming rate caused by the ambient temperature 
that higher than room temperature, and rapp is the cooling 
rate caused by the operation of air conditioner compressor. To 
maintain the set point shift within the allowable set point band 
with T(0) = Tset, we need 



1 



which yields 



S{t)< 



2X, 



(27) 



(28) 



and this is bounded by. 



S{t)< 



i=0 



A^.rAs, 



2T, ' 



(18) 



(19) 



where R{i) is the regulation signal at time i, Tg is the 
temperature gain of air conditioner if it is on, and r is the 
effective thermal constant of the building. 

Proof. The total consumption up to time t is given by, 

t 

Tcooi = tR^^Yl ^(^) = + S{t), (20) 



i=0 



which is an average consumption level of 

S{t) 



Tave — -^b 



t 



(21) 



This is equivalent to choosing a constant non-zero u to 
maintain at a constant consumption level Pave- The relation 
between average consumption and non-zero control is. 



-^ave — 



ton H~ ^off 



N 
/3+n 



c N I N 
a — u /3+n 



iV.^. (22) 



For an uncontrolled process at average consumption, 

(3 

From the above three equations, we have. 
Sit) _ -u 



(23) 



(24) 



t "''a + P 
resulting a constant value of control 

S{t){a + p) 

" = tN^- 

The set point shift after time t by the constant u is given by, 



(25) 



T{t) =T(0) + ir, 



T(0) 

r(o) 



= T(0) + tu5 

Sjt) , N 

= T(0)-^^ 



_ N \ Tband 
^off) 



(26) 



Remark.l According to proposition 1, the accumulated 
long term provision capability is proportional to three pa- 
rameters: A/c,Aset, and r/Tg. The intuition for the first two 
parameters is that a large appliance population and allowable 
set point shift range enables the operator to provide more 
service. As for the parameter r/Tg, we note that a large 
value of r impedes, and Tg facilitates, the change of room 
temperature when the same control is applied. Thus the allow- 
able accumulated provision is proportional to r and inversely 
proportional to Tg. 

When we consider short term regulation, the possible ramp 
of consumption is limited by the state x{t) because we 
are shifting the set point rather than directly turning on/off 
appliances. Aside from x{t), we are interested in finding a 
few fundamental parameters that characterize the short term 
capability similar to those found in the first proposition. The 
second proposition below provides similar results. 

Proposition.2 For a given number of appliances Nc and 
width of temperature band for a specific set point [Tset — 
^Aband^^set + ^Aband]. th^ amount of rescrvc that a building 
can provide for one period ramping is limited by the following 
bounds. 



2N 



^ <AR< TVxn- ^ 



(29) 



r ^band ^ ^band 

where Tg is the temperature gain of air conditioner if it is on, 
and r is the effective thermal constant of the building. 

Proof. In the dynamic operation when the tracking error is 
0, the feedback controller is given by, 

u = \ z[N,{ax^ - /3x2n) - AR]. (30) 

Since the allowable control set is u e [— then AR is 
restricted by 



Nc{a + /3)x2N <AR<Nc{a^ /3)x^ 



(31) 



Using the relation between transition rate and duty cycle 
developed in section |ll| and the fact 

Aband ~ ^off'^off ~ ^on'^on? 

the following inequality is seem to hold: 



^band 



^band 



Using the fact that Ton + ^off = Tg/T yields (29). ■ 

Remark.2 According to proposition 2, the short term pro- 
vision capabiHty is proportional to A/c, and inversely propor- 
tional to Aband and r/Tg. The proportionality to shares 
the same explanation as in the first remark. The two inverse 
proportionalities can be explained by saying that small value of 
Aband makes the width of individual temperature bin smaller, 
and large value of Tg/r makes the thermal transfer faster. 
These two factors in turn facilitate the state transition to 
provide larger one- step service. 

Remark.3 The advantage of taking comfort band Aband ^ 
is that we can provide large value of one step service and 
at the same time make each room temperature stick to the 
set point, with the sacrifice that we shorten the appliance's 
duty cycle. This tradeoff between system performance and 
appliances functioning is consistent with what we find in (91 
where electricity consumption can be smoothed by shortening 
appliances duty cycle. 

Remark.4 The fraction r/Tg affects both long and short 



term regulation provision. Based on ([19]) and ( [29] ), it can be 
seen that a building able to provide large amount of short term 
ramping regulation is more incapable of providing long term 
accumulated regulation. The opposite also holds. Thus the two 
capabilities restrict each other. 

C. Buildings in the Regulation Service Market 

The above propositions determine the maximum regulation 
service that can be provided in the power market. To become 
a qualified provider in the U.S., a building needs to pass the 
T-50 qualifying test fni . Fig. 3 shows the test signal from 
PJM. The dotted line is the ideal consumption response that 
the building consumes according to the test signal. Two key 
requirements are needed to pass the test: 

• Rate of response: the building needs to able to reach the 
maximum or minimum consumption level within 5 minutes. 

• Sustained response: the building needs to able to maintain 
at the maximum or minimum consumption level for 5 minutes. 
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Fig. 3. T-50 test by PJM where regulation providers are obliged to ramp 
their consumption up and down to meet the test requirements. 

The following corollary gives the upper bound on the 
regulation service that a building can provide. 

Corollary. To pass the T-50 qualifying test with a rate of 
response within k minutes {k < 5), the maximum regulation 



service i^r,max that a building can provide is given by. 



20T, 



-, min( 



^on ^oii ) 



(33) 



Proof. For long term regulation service, the maximum value 
of S{t) for t e [0, 50] is S{t) = lORr with t = 30. According 
to proposition 1, 

lORr < (34) 



This yields 



Rr < 



2Tg 

A^crAset 
20T, 



(35) 



For short term regulation service, we need to track the rate of 
consuming up and down starting from steady state. It is easy 
to verify that the steady state probability distribution satisfies, 



Xi 







(36) 



Substitute the above equation into ( [29] ) and note that AR 
Rr/k in a /c minutes ramp process. Finally we have. 



Rr < mm(-^, - — ). 

^on t'off 



(37) 



Taking the minimum of (35) and ( [37] ) completes the proof. 



D. Real Time Spinning Reserve 

When the real time consumption AP ramps up or down 
quickly due to short term stochastic consumption aggregation, 
the ISO has to use spinning reserves to compensate for the 
demand that cannot be covered by the regulation service. 
This follow from the limits of proposition 1 and 2. In such 
cases, a feedback signal from the provider side to the ISO 
is beneficial to the ISO so that it can determine the needed 
spinning reserve Pspin- The following proposition establishes 
the relation between Pgpin and AP when necessary building 
information is given. 

Proposition.3 Denote AP as the stochastic demand ramp 
in one step, then the spinning reserve Pspin needed to maintain 
grid balance is given by. 



spin 



(38) 



(AP- Ai?„,ax)l{AP>AK„„}- 
(AP-Ai?„i„)l{AP<Aii„,„}, 

where l^.y is the indicator function, and 

I (roffXN - rona;2N)+ 
min( , ^on) {xn + :^2n) 

rpmax rj-i ^ 

are the maximum and minimum reserve provision threshold 
for the building. 



AR, 



ARmm 



(39) 



Proof. The allowable control u is limited by two factors. 
The first is that the set point shift resulting from u should be 
within the allowable set point range, 

The second is the requirement of maintaining non-negative 
Markov rate, 

u e [— /3, a]. 



From the above two inequalities 

^set 



u e 



max( 



^set 



AtS 



min( 



^set 



At5 



(40) 



From ([30]), the one period ramping can be expressed in terms 
of u, 

AR = Nc{{ax^ - /3x2n) - ^(^N + ^2n)} (41) 

Substituting ^ into ^ yields AR G [Ai^niin, Ai^max], 
where Ai^min and Ai^max take values in ([39]). For the grid 
balance, we have. 



AP = AR^ P, 



spin 7 



(42) 



and we wish to use as little spinning reserve as possible. Then 
will take value in ([381). The indicator function in ([38l) 



spin 



gives the condition that AP is outside the range of ARM 

In the U.S. regulation service market, the ISO has purchased 
regulation service a day ahead from multiple m providers. 
Assuming that feedback signals are set up between all the 
providers and the ISO such that the latter receives information 
from all of them every 4 seconds. Upon receiving these 
information, the ISO knows their individual instant capability 
range [Ai^^-^^, Ai?^^^], i=l,...,m, and then dispatches regula- 
tion signals to each of them. The signals are dispatched such 
that the minimum spinning generation is used and such that 
it should be within the range of capability for the building 
to respond. Most of the time, the one period ramping is 
within the total regulation capability of the m buildings so 
no spinning generation is needed. In such cases, the ISO can 
have more than one dispatch solution. The question arises that 
how the ISO should dispatch signals in an optimized way. The 
next subsection answers the question by solving a quadratic 
program (QP). 

E. Optimization of Real Time Regulation Signal Dispatch 

When there is more than one solution to the regulation 
dispatch at time t such that no spinning generation is needed, 
an optimal way to dispatch is that we maximize the regu- 
lation provision capability at time t + 1. From proposition 
3, the i^^ building can provide regulation service with range 
AR e [ARi^^, ARi^^J. The width of the closed interval of 
regulation service is given by 



ARi 



min( 



T:,,{t+i)-T:^ 



min(— ^ 



(43) 



AtS 



Then the objective function is given by 



max^WJ - MP' 



,2 

spin' 



(44) 



namely we are maximizing the sum of m regulation service 
widths from each building at time t + 1, minus the spinning 
generation penalty Pspm with positive coefficient M. The 
maximization problem is subject to the following type of 
constrains: 

State Dynamics: 



4(t + 1) = + At{a' - u'){xi^_,{t) - 4(t)), 
xl^{t + 1) = xl^{t) + At{(3^ + ^^)(4n-i W - 4nW)- 



(45) 



Feedback Controller: 



N'^At{xl,{t)^xi^{t))u'^AR' = N'^At{axi,{t)-/3xi^{t)). 

(46) 

Supply-demand Balance: 



Pspin — A P. 



Non-negative Markov Rate: 

/3' <u' < a\ 
Allowable Regulation Range: 

Rh- Rr <Cx^ AR' < + 
Allowable Set Point Range: 

C/"'' < V^tit + 1) = T^t) + u'AtS < t: 



max,* 
set 



(47) 



(48) 



(49) 



(50) 



The above optimization cannot be solved with standard tech- 
nique because the objective function has the minimum opera- 
tor. We transform the original problem into a QP. Let 



T« (f - 7^min,i 

™l min^ setV'' ^ set „J ^ 

nil — >^onj> 



i| = min(- 



■TUt+1) 



(51) 



AtS ' 
Then the original objective function becomes 

m 

max N^{xi^{t + 1) + xi^{t + l)){m{ + mi) 

i=l 

-MP'^- . 

spin* 

If we add the following constrains, 

^on? 

r;,(^ + i)-cr 



(52) 



m\ < Ton, 



m\ < 



(53) 



ml < Toff, 

r,T-' - T^t + 1) 



ml < — ^ 



At5 



and solve the QP, 



max (52) 



s.t. (45) - (50), (53) 



(54) 



we will obtain the same optimal solution as solving the 
original problem. This is because when reaching the optimal 
solution, one of the inequality constraints for both m\ and 



ml in (53) will be strict, otherwise the optimal solution is 



not reached since we can increase the value of m\ or ml 
to increase the value of objective function due to positive 
coefficient N^{xi^{t + 1) + 4n(^ + 1)) in Then (IsT) 



is satisfied and optimizing over (54) is equivalent to solving 
the original problem. Note that (54) has quadratic objective 
function with linear constraints, Matlab or CPLEX can solve 
this QP efficiently. 

IV. State Observer Design 



In controller design (17), it is assumed that x{t) can be 
measured. If this is not true, especially when the temperature 
band is finely discretized, i.e. 5 is small so that the sensor 
cannot provide the required precision, we need to design 
an observer to estimate the state. Considering the following 
dynamics of the observer. 



Ax + Bxu + L{t){y - Cx), 



(55) 



where L{t) is the time varying column vector to be designed. 
The last term is similar to the innovation term in Luenberger 
filter 115]. Define the estimation error, e = x — x, then 



e = {A^Bu-LC)e. 



(56) 



In the observer design, we restrict the allowable control set to 
prevent control saturation with some small constant positive 
margin e > 0, 



u{t) e [-(3^e<u{t) <a-e]. 



(57) 



This guarantees an irreducible and ergodic Markov chain. 
If the control signals is allowed to stay saturated, then the 
Markov chain breaks and some of the states become isolated, 
see Fig. 1(a) when you substitute u = — /3 or u = a into 
the model. Estimation error of those isolated states cannot be 
guaranteed. 

The following proposition states that we can select L{t) 
such that the estimation error will approach zero as t ^ oo. 



Proposition.4 If the control is chosen according to ( [57] ), 
then the following holds: 

(1) There exists a time varying observer design L{t) = 
[0, . . . ,0,L2n(^)]^ such that 



2Ar-i 



e^{A^Bu-LC)e < -e(t)||e||^ 



(58) 



for all e{t) with e{t) = j mm[P ^ u{t) , a - u{t)], 7 G (0, 1). 

(2) The estimation error would converge to zero asymptot- 
ically 

lim e{t) = (59) 



Proof. (1). To prove ( [58] ), it is equivalent to show that 

e^{A ^Bu-LC^ ^it)I)e < 0. (60) 

Denote A = A-\-Bu — LC-\-e{t)I, then A{t) can be expressed 
by the following equation based on the special structure of the 
state space matrices A, B and C derived in section [ll] 



-a -\- u 



/3 + n \ 



-a + u 



\ 
/ <t) 







-/3 -u 




-/3 -u 
+ u 





-(3 -u J 



e(t) 




V 



e(t) 




e{t) 
N^L2N e{t) - N^L2N- / 



(61) 



Denote Ai as the i—hy—i square matrix from the upper left 
part of A. In order to show that A is negative definite, it is 
equivalent to show that 

(-1)^ dei{Ai) > 0, for z = 1, . . . , 2N . (62) 

For i = 1,...,2A^ — 1, Ai is a triangular matrix whose 
determinant is the product of its diagonal elements. From (61 
the diagonal elements Ai^i is given by. 



J —a + u 

{ -i3-u 



+ e(t) ifi = l,...,N 



^ ^ -. + e(t) if i = 7V + l,...,2Ar-l 



(63) 



Since e{t) = 7min[/3 + u{t)^a — u{t)], we have Ai^i < (7 — 
1) min[^ + u{t), a - u{t)] < for z = 1, . . . , 2Ar - 1. This 
yields 

i 

{-ly det(ii) = (-1)^ Yl ^jJ > 0, for i = 1, . . . , 2^- - 1. 

i=i 

(64) 

As for i = 2N, det{A) is a linear function of L2N as the 
observer parameter only appears in the last row of the matrix, 
so there must exists a L2N such that det(74) > 0. This 
completes the proof of negative definiteness of the matrix A, 
and ([58]) holds. 
(2). Let 

/e(r)dr 

y{t) = eo \\e{t)\\\ (65) 
then the time derivative of y{t) is given by, 

f e(T)dT _ ~ f e(T)dT 

y{t) =e(t)eo \\e{t)f +2e^{t)Ae{t)eo 
= Jo'^^^'"'\e{t)\\e{t)f + 2e^{t)Ae{t)] 



fe{T)dT 
< go 



eW||e(t)f -2e(t)||e(t)||2 



= -ei e(t)||e(t)||2 



(66) 



The time derivative from the above equation indicates that y{t) 
in ([65l) is stable at zero. 



Maximum reserve level 0.25, duty cycle 5 n 



/e(r)dr 

On the other hand, from i65\ we have lim =00 

t^co 

and lim y{t) = 0. Necessamy we need lim e{t) = to 
satisfy these two conditions, which indicates that the error 
will approach zero asymptotically. ■ 

Remark.5 e{t) = 7min[/3 + u{t)^a — u{t)] > is the 
lower (conservative) bound of the convergence rate we can 
achieve. In real time operation when the control signal is 
distributed across the allowable set to be away from saturation, 
the numerical convergence rate is larger than the conservative 
bound. It should be noted that as the duty cycle increases, 
the boundary value of a = ^ and f3 = ^ decreases, 
which means we have smaller allowable control set. Then 
the value of control signal would be closer to the boundary 
than the scenario with small duty cycle appliances, therefore 
it is anticipated that the numerical convergence speed of the 
error vector will be slower for system having large duty cycle 
appliances. 



V. Simulation 

In this section, we provide three simulation to verify the 
theoretical framework proposed in previous sections. In the 
first example, we will show how building parameters derived 
in the first two propositions restrict the maximum delivery 
of regulation service. In the second one, we will verify the 
effectiveness of the proposed QP optimization in regulation 
signal dispatch by reducing real time spinning generation. In 
the last example, we will show the performance of the observer 
that asymptotically reduces the estimation error. 



A. Long and Short Term Regulation Service Limitation 

Fig.4 shows how the long and short term restrictions on 
regulation service affect the T-50 test. The first figure is 
an example of the inability to provide enough accumulated 
regulation service due to limited allowable set point choice. 
In this figure, the set point shift hits the lower bound of the 
allowable set between 15 to 20 minutes. As a consequence, the 
building cannot provide a sustained high consumption level by 
further decreasing the set point. Similarly, the set point shift 
hit the upper bound after 40 minutes which restricts the sustain 
provision of lower level aggregated consumption. 

The second figure is an example to show the inability to 
provide short term regulation service. Although the set point 
shift is within the allowable set, the ramping speed to provide 
positive and negative regulation service is smaller than the 
required speed for a given maximum regulation obligation. 
This is because the thermostatic appliances in this simulation 
have large duty cycle. For example, the typical duty cycle 
of refrigerators is around 20 minutes and has large effective 
thermal constant 1 17 |. Such a large duty cycle greatly restricts 
the maximum ramping rate in regulation service derived from 
proposition 2. 




(a) Long Term Provision 



Maximum reserve level 0.25, duty cycle 25 min 




(b) Short Term Provision 

Fig. 4. Individual providers fail to pass the T-50 qualifying test due to either 
a bounded allowable set point range or a limited ramping capability. 



B. Optimal Regulation Signal Dispatch 

We use real time PJM data |[T6ll to verify the performance of 
the overall two level control and optimization framework. We 
can see that the regulation signal is stochastic and ramps up 
and down according to the area control error with no apparent 
probability distribution. Fig. 5 shows the simulation results of 
the proposed system with lower level feedback linearization 
controller design and higher level two-way communication 
feedback design. Fig. 6 is the simulation result with only lower 
level controller. We find that both systems can track the 
regulation signal when it ramps up or down with the design 
of building level feedback controller, but they differ in the 
maximum ability of tracking. Compared with Fig. 6, where 
individual building regulation signals are proportional to their 
maximum provision level (ISO does not have knowledge about 
real time building information, so it dispatches signals propor- 
tional to their a prior maximum commitment), the signals in 
Fig. 5 are determined by solving the QP. We find that the 
consumption trend of red curve and blue curve are different 
most of the time in Fig. 5 that fully exploits their tracking 
capability, while in Fig. 6 individual building consumptions 
follow the same trend with proportionality. The accumulated 
generation of spinning reserves is reduced when the regulation 
dispatch is optimized by the QP, and the distribution of 
spinning reserve is more concentrated around with smaller 
variance. Tab.l shows the statistics of both systems, the total 
spinning generation is reduced by 50% after optimization. 



The standard deviation, maximum, and minimum spinning 
generation is reduced by 30%, 30%, and 15% respectively. 
Furthermore, at times when the regulation signal ramps up 
and down at a fast speed, for example at 15, 30, 35, 40, 
and 50 minutes, the performance of the two level structured 
system in Fig. 5 is better than the system performance without 
the design. This is because the required spinning reserves is 
largely reduced despite the sharp ramping regulation signal. To 
sum, the proposed two level feedback framework outperforms 
the system with only building level controller design. 
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(b) Spinning Generation 

Fig. 6. Real time regulation dispatch when the ISO does not receive 
information from individual providers. Spinning generation is more spread 
out around with larger variance. 



Spinning Generation Distribution 




(b) Spinning Generation 

Fig. 5. Real time regulation dispatch after the quadratic program is 
solved. Tracking becomes close to the target and spinning generation is more 
concentrated around with smaller variance. 



norm converges to zero after 20 minutes when the duty cycle 
is 10 minutes, and converges to zero after 35 minutes when the 
duty cycle is 20 minutes. This is because smaller duty cycle 
enables a larger allowable control set, i.e the system has large 
values of a and (3 to prevent from saturation. In Fig.7(c) and 
Fig. 7(d) we compare the distribution of control signals across 
the allowable control set, it can be seen that the control signal 
stays further away from saturation for system having smaller 
duty cycle. Such systems have a larger convergence speed of 
the error vector. 



TABLE I 

Comparison of System Statistics 



Unit/kW 


Spin. Gen. 


Mean 


Std. Dev. 


Max 


Min 


Opt. Alg. 


10216 


0.18 


27.20 


100.27 


-210.59 


Prop. Alg. 


20525 


-0.44 


41.97 


146.19 


-247.65 



C. Observer Performance 

We verify the performance of the proposed observer. 
Fig. 7(a) shows the real and estimated value of two sample 
states, and X2n, these are the two states around the 
boundary of the comfort band that will affect the change in 
electricity consumption. We can see that the estimation error 
for these two boundary states approach zero after 20 minutes. 
Fig. 7(b) shows the convergence speed of the norm decay of the 
error vector e{t) for appliances with different duty cycles. The 



VI. Conclusions and Future Work 

This paper proposes an innovative two level feedback design 
system for the real time regulation provision. The lower level 
building feedback controller allows the building operator to 
track a given signal from the ISO, and the higher level infor- 
mation feedback from individual buildings to the ISO allows 
the latter to optimally dispatch real time regulation signals to 
multiple providers by solving a quadratic programming prob- 
lem such that the cost of spinning reserves is reduced. During 
the development of the system, we also derive analytically 
the building's limitation on both long term and short term 
regulation provision and provide intuitive explanations that 
match our theoretical results. To counter the problem with 
effective measuring, we also proposed an observer design to 
reduce the error of state estimation such that the feedback 



controller can work well. Simulation results indicate that the 
proposed framework is superior in reducing the amount of real 
time spinning reserve. 

Future work will include the modelling of similar two level 
feedback system but with price based signalling between the 
operator and the consumers. Such design will fit into problems 
where consumers do not authorize the operator to directly 
control their set point. It is expected that price-based mecha- 
nism has looser control compared with the direct load control 
because consumers can choose their own utility function. 
Consequently, it enables consumers to obtain different level 
of desired comfort. 
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Fig. 7. Observer performance validation, (a) Real and estimated value of 
two sample states and X2n- (b).The norm of the estimation error gradually 
approaches zero as the estimation proceeds. Convergence rate of the estimation 
error vector depends on the duty cycle value, (c) and (d) the distribution of 
control signal across the allowable set when different duty cycle values are 
used 



